The present invention subject matter relates to methods and systems for reconstructing 3-dimensional objects from sequences of 2-dimensional images. It finds particular application in conjunction with the estimation of camera motion parameters from data obtained with a moving camera, and will be described with particular reference thereto. However, it is to be appreciated that it is also amenable to other like applications.
An increase in quality, coupled with a decrease in price of digital camera equipment has led to growing interest in reconstructing 3-dimensional objects from sequences of 2-dimensional images. Further, the ready availability of high quality and low price digital cameras has led to the development of models and systems that allow the capture of accurate 3-D spatial data from a sequence of 2-D images. One approach has been to collect the sequence of 2-D images from an object space by moving the camera along a predetermined path. Using the image sequence and the concepts of triangulation and parallax, 3-D spatial data from the object space may be recovered.
The quality of the 3-D reconstruction of the object space is dependent on many factors. Among them are resolution of the sensors used, lighting conditions, the object details, and calibration errors. There are several sources that contribute to calibration errors such as, e.g., inaccuracies inherent in the camera lens, including inaccuracies in the lens specifications, and inaccuracies in the means used to move the camera along the desired path. Therefore, it is essential for proper calibration, to estimate error introduced by camera movement, and to provide algorithms to remove or compensate for this error. This process is referred to hereinafter as moving camera calibration.
Moving camera calibration consists primarily of two steps. The first step is to use a sequence of images to estimate the 3-D motion parameters of a moving camera, and the second step is to design a filter to correct the error between the desired motion and the estimated motion. The problems of camera parameter estimation have been addressed by a wide range of researchers. Their approaches have been successful in systems with little or no noise. However, in most cases, the 3-D transformation has been modeled as a nonlinear stochastic system, and it is necessary to estimate the state variables from noisy observations.
There are several sources that contribute to observation noise, including camera motion, projection noise, and/or random disturbances from a moving object. To solve the observation noise problem, a Kalman filter (IEKF) has been widely used as a nonlinear estimator. For example, Denzier and Zobel use a Kalman filter to estimate camera parameter with a selected focal length (On optimal camera parameter selection in Kalman filter based object tracking, by J. Denzler, M. Zobel and H. Niemann, Pattern Recognition, 24th DAGM Symposium, Zurich, Switzerland, pp. 17-25, 2002). Koller and Klinker use an extended Kalman filter to estimate the motion of the camera and the extrinsic camera parameters (Automated camera calibration and 3D egomotion estimation for augmented reality applications by D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker and M. Tuceryan, 7th International Conference on Computer Analysis of Images and Patterns (CAIP-97), Kiel, Germany, Sep. 10-12, 1997). Goddard and Abidi use dual quaternion-based iterated extended Kalman filter to estimate relative 3-D position and orientation (pose), (Pose and motion estimation using dual quaternion-based extended Kalman filtering by J. S. Goddard, M. A. Abidi, A Dissertation Presented for the Doctor of Phiosophy Degree, The University of Tennessee, Knoxville). Venter and Herbst use an unscented Kalman filter to estimate the motion of an object from a video sequence under perspective projection (Structure from motion estimation using a non-linear Kalman filter by C. Venter, B. Herbst, Dept. of Electronic Engineering and Dept. of Applied Mathematics, University of Stellenbosch, 7600, South Africa). The above-described references are incorporated herein by reference.
The accuracy of camera motion models depends primarily on two sets of parameter estimates. The first set of parameters includes lens parameters such as, e.g., focal length, principal point, and distortion parameters. The second set of parameters includes a set of motion parameters that enables the comparison of a moving camera's theoretically determined physical location to a desired location.
The present invention is directed toward improving the accuracy of the second set of parameters, i.e. the estimation of the set of 3-D motion parameters from data obtained with a moving camera. A method is provided that uses Recursive Least Squares (RLS) for camera motion parameter estimation with observation noise. This is accomplished by calculation of hidden information through camera projection and minimization of the estimation error. The present invention also provides a method for designing a filter, based on the motion parameters estimates, to correct for errors in the camera motion.